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Foreword 


This  report  describes  the  four  major  components  of  organizational  effectiveness  (Productivity, 
Financial  Performance,  Stakeholder  Relations,  and  Resource  Development)  and  provides 
examples  of  measures  that  may  be  useful  for  organizations  that  are  contemplating  the 
implementation  of  a  Productivity  Gain  Sharing  (PGS)  system. 

The  success  of  PGS  requires  measures  that  accurately  reflect  improvements  in  productivity  and 
other  components  of  an  organization’s  performance.  Thus,  organizations  need  to  have  a  valid  and 
reliable  performance  measurement  system.  This  report  demonstrates  how  effective  performance 
indicators  can  be  developed. 

This  report  is  one  of  a  series  of  reports  on  measurement  issues  related  to  PGS  and  total  quality. 
The  other  reports  in  this  series  are:  (1)  Using  Performance  Indexing  to  Measure  Organizational 
Gains  in  a  White  Collar  Environment  (Tatum,  Nebeker,  &  De  Young,  1996);  (2)  An  Approach  to 
Measurement  of  Quality  and  Productivity  for  Gain  Sharing:  Measuring  Total  Organizational 
Value  (Nebeker  &  Tatum,  1996);  and  (3)  Integrating  Measurement  Approaches  in  Gain  Sharing 
and  Total  Quality  (Tatum,  Shaw,  &  Main,  1996). 

This  report  also  has  a  companion  report  that  discusses  many  of  the  components  of  organizational 
effectiveness  in  greater  detail.  The  companion  report  is  A  Typology  of  Organizational  Effectiveness 
(Tatum,  Nebeker,  &  Wolosin,  1996). 
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Summary 


Problem  and  Background 

The  Federal  Government  has  recognized  the  need  for  increased  organizational  effectiveness 
within  its  agencies.  Organizational  effectiveness  will  become  more  of  a  necessity  in  the  future  with 
the  implementation  of  the  Government  Performance  and  Results  Act  of  1993.  However,  it  has  been 
a  problem,  especially  for  white  collar  organizations,  to  measure  performance  accurately.  This 
poses  a  particular  problem  for  those  organizations  that  are  contemplating  the  implementation  of  a 
Productivity  Gain  Sharing  (PGS)  system.  The  success  of  PGS  requires  an  accurate  reflection  of 
organizational  performance. 

Objective 

The  purpose  of  this  report  is  to  demonstrate  how  effective  performance  indicators  can  be 
developed  to  establish  a  valid  and  reliable  measurement  system.  It  is  to  be  used  as  a  primer  on  the 
theory  and  practice  of  performance  measurement  for  all  organizations,  but  especially  for  white 
collar  organizations  interested  in  developing  measures  of  organizational  effectiveness. 

Approach 

The  report  first  discusses  basic  measurement  issues.  Second,  it  breaks  down  organizational 
effectiveness  into  four  major  components  (Productivity,  Financial  Performance,  Stakeholder 
Relations,  and  Resource  Development).  Each  component  is  then  discussed  and  examples  of 
measures  are  provided  that  may  be  useful  for  the  purpose  of  PGS. 

Conclusions 

Organizations  interested  in  increasing  their  effectiveness  need  to  establish  valid  and  reliable 
performance  measurement  systems.  This  is  especially  true  for  those  organizations  that  would  like 
to  implement  PGS,  because  the  success  of  PGS  rests  on  the  quality  of  the  measurement  system. 

When  developing  a  measurement  system,  emphasis  should  be  placed  on  total  organizational 
improvement  and  not  just  limited  to  improvement  in  one  area.  Effective  PGS  systems  should 
incorporate  some  measures  from  all  four  components  so  that  total  organizational  performance  can 
be  monitored  and  improved. 
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Introduction 


This  document  represents  a  primer  on  the  theory  and  practice  of  performance  measurement.  As 
the  title  suggests,  the  principle  audience  is  the  government  organization,  but  almost  any 
organization  (service,  manufacturing,  technical,  government,  etc.)  can  benefit  from  the  general 
measurement  principles  outlined  in  this  paper.  Although  this  report  will  be  useful  for  any 
organization  concerned  with  accurate  and  reliable  performance  measurement,  the  information  here 
will  be  especially  useful  for  organizations  contemplating  the  implementation  of  productivity  gain 
sharing  (PGS).  The  success  of  PGS  depends  largely  on  the  precision  and  validity  of  the 
measurement  system,  and  this  document  provides  guidance  on  how  to  develop  measures  of 
performance.  For  government  organizations,  even  if  PGS  is  of  no  interest,  accurate  measurement 
is  going  to  become  more  of  a  necessity  in  the  future  with  the  implementation  of  the  Government 
Performance  and  Results  Act  of  1993. 

Developing  a  measurement  system  involves  several  steps  as  outlined  by  Tuttle  and 
Sink  (1984).  Two  critical  steps  involve  defining  key  result  areas  and  developing  performance 
measures  (see  Tatum  and  Nebeker,  1996,  for  a  description  of  these  and  other  measurement  steps 
using  software  development  as  an  illustration).  An  organization’s  key  result  areas  are  absolutely 
essential  to  meeting  its  mission  objectives  and  strategic  goals  (see  Tuttle,  Wilkinson,  &  Mathews, 
1985,  for  excellent  examples  of  the  process  used  to  generate  key  result  areas).  Performance 
indicators  measure  these  key  result  areas.  Organizations  interested  in  PGS  can  use  this  document 
to  (1)  help  define  key  result  areas;  (2)  identify  performance  indicators  for  PGS;  (3)  begin 
measuring  the  indicators  and  tracking  them  over  time;  (4)  revise,  eliminate,  or  add  new  indicators, 
(5)  decide  on  the  optimal  set  of  indicators  for  PGS;  and  (6)  make  future  adjustments  to  the 
indicators. 

PGS  focuses  on  improving  the  total  performance  of  the  organization.  Measures  of  total 
organizational  performance  (organizational  effectiveness)  fall  into  four  broad  areas,  productivity, 
financial  performance,  stakeholder  relations,  and  resource  development,  as  shown  in  the  typology 
presented  in  Table  1.^  Each  of  these  areas  can  be  further  subdivided  into  more  narrowly  focused 
areas  (also  shown  in  Table  1).  This  document  will  discuss  each  of  these  areas  briefly  (for  a  more 
detailed  treatment  see  Tatum,  Nebeker,  &  Wolosin,  1996)  and  then  make  recommendations  for 
developing  specific  measures  that  reflect  improvements  in  these  areas  (some  sample  measures  are 
also  shown  in  Table  1).  Not  all  organizations  will  be  interested  in  all  of  the  measures  discussed. 
Certain  areas  and  measures  will  be  relevant  to  some  organizations,  while  other  areas  and  measures 
will  be  more  germane  to  other  organizations.  Probably  some  measures  are  desirable  from  each  of 
the  four  broad  areas,  but  for  the  purposes  of  PGS  most  of  the  measures  will  come  from  the  first  two 
areas  (productivity  and  financial  performance).  The  underlying  purpose  of  this  document  is  to  get 
organizations  to  think  about  what  the  most  appropriate  set  of  measures  should  be  for  them,  and  then 
develop  that  set  to  track  improvement  and  establish  a  measurement  system  for  PGS.  Although  the 
measures  described  here  focus  on  the  needs  of  white  collar  organizations,  many  of  the  measures 
will  apply  to  any  type  of  organization. 


*Table  1  is  a  typology  rather  than  a  taxonomy.  Taxonomies  are  empirically  derived,  hierarchical  systems  whereas 
typologies  are  classifications  based  on  theoretical  and  intuitive  categories  (see  Rich,  1993,  for  more  details  on  the 
distinction  between  taxonomies  and  typologies). 
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Examples  of  Measures  Using  the  Organizational  Effectiveness  Typology 

Organizational  Effectiveness 

Productivity  Financial  Performance  Stakeholder  Relations  Resource  Development 
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Timeliness 

1.  Just-In-Time  Index 

2.  Shelf  life 

3.  Conformance  to  schedule 


Basic  Measurement  Issues 

Before  discussing  specific  measures,  some  basic  issues  related  to  measurement  will  be 
introduced. 

Criteria  for  Good  Measures 

If  an  organization  wants  to  select  the  best  set  of  performance  measures,  several  criteria  should 
be  met.  Many  of  the  desirable  characteristics  can  be  described  with  the  acronym  ACORN. 

A  =  Accomplishment — the  measures  should  reflect  the  accomplishments  of  the  organization 
and  not  just  behaviors  that  may  or  may  not  be  of  value  to  the  organization. 

C  =  Control— ih&  measures  should  be  under  the  control  of  members  of  the  organization. 

O  -  Overall — ^the  measures  should  reflect  important  elements  of  the  organization  as  a  whole. 
They  should  cover  all  critical  aspects  of  organizational  performance. 

R  =  Reconciled— tho,  measures  should  be  reconciled  with  the  objectives  of  all  organizational 
elements.  The  entire  set  of  measures  should  be  consistent  with  the  optimal  performance  of  the 
whole  organization.  In  other  words,  individual  measures  that  reflect  a  part  of  the  system  should  not 
be  emphasized  at  the  expense  of  the  whole. 

N  =  Numeric — ^the  measures  should  reflect  a  numerical  value  that  can  be  monitored  across 
time. 

In  addition  to  having  the  above  characteristics,  measures  should  also  have  the  following 
properties  if  they  are  to  be  truly  useful  for  the  organization. 

Validity 

Validity  refers  to  whether  a  measure  accurately  reflects  the  characteristic  it  is  intended  to 
capture.  Validity  is  the  most  important  property  of  a  measure.  Mean-time-between-failures  of  a 
part,  or  how  closely  a  product  meets  its  specifications,  are  examples  of  valid  measures  of 
performance  because,  usually,  they  accurately  reflect  aspects  of  quality.  But  how  do  you  measure 
the  quality  of  some  product  or  service  that  is  not  so  easily  quantified,  such  as  an  audit,  or  a  training 
program,  or  a  research  project?  In  such  cases,  more  subjective  measures  must  be  developed  and 
used.  However,  this  is  not  necessarily  bad,  because  subjective  measures  are  often  perfectly  valid 
indicators  of  the  characteristic  being  measured.  For  example,  customer  satisfaction  ratings  of  a 
service  are  usually  the  best  indicator  of  the  quality  of  a  service.  Indeed,  subjective  data  such  as 
satisfaction  ratings,  if  collected  properly,  can  have  greater  validity  than  more  objective  data  (e.g., 
sales  have  been  shown  to  be  a  poor  indicator  of  a  product’s  quality). 

Reliability 

Reliability  refers  to  the  consistency  of  a  measure.  A  useful  measure  remains  stable  across  time. 
Even  hard  data  (e.g.,  items  shipped,  engines  repaired)  may  not  have  this  property.  If  the  attribute 
being  measured  is  inherently  stable,  but  the  measurement  of  that  attribute  fluctuates  greatly  (e.g.. 
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due  to  inadequate  data  recording  procedures),  then  the  information  may  be  too  inconsistent  for  the 
organization  to  use,  no  matter  how  objective  it  is.  If  measures  are  not  reliable,  then  efforts  must  be 
taken  to  make  them  more  stable  (e.g.,  calibrate  data  recording  instruments,  collect  the  data  for 
longer  time  periods,  aggregate  several  sources  of  data  into  a  single  measure). 

Standardization 

Standardization  refers  to  collecting  measurement  data  in  a  consistent,  uniform  fashion. 
Standardization  helps  to  achieve  greater  reliability  and  insures  that  errors  are  not  introduced  into 
the  measurement  system. 

Sources  of  Measurement 

An  organization’s  performance  can  be  derived  from  several  sources. 

Objective  Sources 

Most  organizations  have  systems  that  generate  quantifiable  performance  data.  For  example,  if 
an  organization  makes  machine  parts,  or  repairs  engines,  or  delivers  supplies,  it  probably  has 
systems  in  place  (e.g.,  automated  tracking,  manual  record  keeping)  that  monitor  both  the  processes 
and  outputs.  The  measurements  that  are  routinely  taken  from  these  systems  can  be  used  as 
measures  of  organizational  performance. 

Subjective  Sources 

Some  types  of  information  cannot  be  obtained  by  objective  means.  For  example,  a  customer’s 
level  of  satisfaction  with  a  product,  or  the  quality  of  work  performed  by  a  research  team,  are 
inherently  a  subjective  judgments.  Many  important  elements  of  an  effective  organization  can  be 
assessed  only  by  pooling  expert  judgments  or  surveying  public  opinion.  These  subjective  measures 
are  not  necessarily  less  desirable  measures  than  the  objective  measures  discussed  above.  If 
measures  are  valid,  reliable,  and  standardized,  it  really  does  not  matter  if  they  come  from  objective 
or  subjective  sources.  In  fact,  if  the  procedures  for  obtaining  subjective  judgments  are 
standardized,  these  judgments  may  be  considered  objective  to  those  collecting  the  information.  An 
employee  survey,  for  example,  if  conducted  properly,  can  tell  an  organization  a  great  deal  about 
its  climate  or  culture  despite  the  fact  that  the  source  of  the  data  are  subjective  opinions  or 
perceptions. 

Archival  Sources 

Most  organizations  maintain  archival  records  of  many  aspects  of  their  operations.  Records  of 
sick  leave  use,  absenteeism  employee,  turnover,  or  accident  rates  are  usually  recorded  by 
organizations.  These  archival  data  can  often  be  used  as  indices  of  organizational  performance.  For 
example,  if  an  organization  suspects  that  productivity  suffers  because  of  high  absenteeism,  then  an 
examination  of  the  archival  records  can  verify  this  suspicion.  Collecting  archival  data  on 
absenteeism,  or  turnover,  or  accident  rates  can  be  useful  in  determining  whether  the  organization 
is  improving  its  performance. 
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Types  of  Performance  Measures 

The  Department  of  Defense  (DOD)  Comptroller  has  issued  guidance  on  measuring 
organizational  performance  and  has  outlined  a  useful  classification  of  performance  measures 
(Memorandum  from  Acting  DOD  Comptroller,  1992).  According  to  the  DOD  Comptroller, 
performance  measures  are  indicators  of  an  organization’s  effectiveness  and  efficiency  that  are 
directly  tied  to  results.  Performance  measures  document  the  relationships  between  resources 
(inputs)  and  results  (outcomes)  as  illustrated  in  the  following  diagram: 


INPUTS - ►PROCESSES - ►  OUTPUTS - ►  OUTCOMES 


According  to  the  DOD  Comptroller,  performance  measures  fall  into  four  categories:  efficiency, 
effectiveness,  outcome,  and  process. 

Efficiency  Measures 

Efficiency  measures  demonstrate  how  well  an  organization  is  using  its  resources.  They  are 
typically  expressed  as  an  input/output  ratio.  For  example,  unit  cost,  labor  productivity,  and  cycle 
time  are  aU  classified  as  efficiency  measures  because  they  relate  some  input  (cost,  labor  hours, 
time)  to  some  output  (e.g.,  items  shipped,  parts  repaired,  students  trained).  Efficiency  is  often 
referred  to  as  “doing  things  right”  because  it  tells  an  organization  to  what  degree  it  is  using  its 
resources  wisely. 

Effectiveness  Measures 

Effectiveness  measures  mean  “doing  the  right  thing.”  They  indicate  whether  an  organization  is 
meeting  its  output  goals.  These  goals  can  be  specified  along  a  quantitative,  qualitative,  or 
timeliness  dimension.  For  example,  if  a  goal  is  to  repair  a  certain  number  of  engines  in  a  given 
month,  a  quantitative  index  of  effectiveness  would  reflect  whatever  that  goal  was  met  or  exceeded. 
Another  measure  of  effectiveness  might  indicate  that  a  product  or  service  exceeded  objective 
criteria  for  quality,  or  that  an  organization  met  scheduled  completion  dates.  In  all  of  these 
examples,  effectiveness  is  judged  by  how  well  an  organization  meets  its  goals  for  quantity,  quality, 
or  timeliness.  Measures  of  effectiveness  do  not  have  to  be  strictly  objective  measures  such  as 
meeting  a  production  quota  or  deadline.  Customer  satisfaction,  for  example,  is  a  subjective 
judgment  made  by  the  user  of  the  product  or  service.  Nevertheless,  customer  satisfaction  ratings 
are  clearly  an  important  measure  of  the  effectiveness  of  an  organization. 

Process  Measures 

Process  measures  capture  the  way  work  gets  done  during  the  process  of  producing  a  good  or 
delivering  a  service.  Process  measures  indicate  the  degree  of  control  over  internal  operations. 
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These  measures  can  be  used  to  diagnose  problems  and  streamline  work  procedures.  For  example, 
processing  a  delivery  order  in  a  government  contract  office  requires  several  steps  (e.g.,  write 
statement  of  work,  develop  cost  estimate,  obtain  technical  review,  make  request  for  quote, 
negotiate  with  contractor).  Each  of  these  steps  or  processes,  in  turn,  can  be  broken  down  into 
subprocesses,  and  the  subprocess  can  be  broken  down  into  sub-subprocess,  ad  infinitum.  Any  or 
all  of  these  processes  and  subprocesses  can  be  measured  and  these  measures  can  be  used  to 
improve  efficiency.  For  example,  if  the  contracting  office  measures  the  time  required  to  complete 
each  step,  these  times  can  be  used  to  diagnose  where  the  bottlenecks  are  in  the  system  (e.g.,  writing 
a  statement  of  work  takes  longer  than  the  other  process).  If  changes  are  made  to  streamline  part  of 
the  process,  the  process  measures  can  help  verify  whether  these  changes  had  a  positive  effect  (i.e., 
training  on  how  to  write  a  statement  of  work  reduced  the  time  by  50%).  The  variability  in  process 
measures  can  also  be  indicative  of  improvements.  For  example,  training  people  to  write  a  statement 
of  work  may  reduce  the  variation  in  writing  times  from  contract  to  contract  which  is  a  reflection  of 
a  more  stable,  more  consistent  process.  Process  measures  help  us  improve  products  and  services 
by  making  the  production  or  delivery  process  more  efficient.  However,  process  measures  do  not 
measure  the  attributes  of  the  products  and  services  directly.  Process  measures  are  critical  if  an 
organization  is  striving  for  continuous  performance  improvement. 

Outcome  Measures 

Outcome  measures  reflect  progress  toward  strategic  goals  or  mission  objectives.  For  example, 
if  the  organization  has  a  strategic  plan  that  specifies  certain  strategic  goals  (e.g.,  health  status  of  its 
employees,  material  readiness),  outcome  measures  can  be  used  to  determine  whether  these  goals 
are  being  met.  As  another  example,  if  the  organization’s  mission  is  to  conduct  research  and 
development  on  weapons  systems,  outcome  measures  would  indicate  how  well  they  were 
performing  their  mission. 

Results-Oriented  Versus  Process-Oriented  Measures 

A  simpler  way  to  classify  measures  is  to  use  two  broad  categories  that  overlap  with  the  DOD 
Comptroller’s  classification  described  above.  The  first  category  represents  measures  that  are  very 
general  in  nature  and  tend  to  be  results-oriented.  This  category  overlaps  with  the  Comptroller’s 
efficiency,  effectiveness,  and  outcome  measures.  Examples  of  such  measures  include  total  hours 
worked,  total  revenue  generated,  units  of  energy  consumed,  cost  associated  with  waste,  or  average 
customer  satisfaction  rating.  These  measures  provide  a  general,  overall  picture  of  how  the 
organization  is  performing  in  terms  of  the  results  it  is  trying  to  achieve. 

The  second  category  represents  measures  that  are  more  specific  in  nature  and  tend  to  be 
process- oriented.  This  category  overlaps  with  the  Comptroller’s  process  measures.  The  measures 
from  this  second  category  are  very  specific  indicators  of  the  internal  processes  of  the  organization. 
Examples  of  this  second  category  include  data  from  control  charts,  measures  of  scrap  or  waste, 
estimates  of  process  capability,  and  so  forth.  The  first  category  of  measures  is  useful  for  getting  a 
global  view  of  an  organization’s  performance.  The  second  category  of  measures,  by  contrast, 
provides  a  more  detailed  view  of  the  organization  and  helps  diagnose  internal  problems  that 
ultimately  affect  the  final  results. 
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Both  results-oriented  and  process-oriented  measures  are  important  to  PGS.  Because  PGS  is 
concerned  with  improving  total  organizational  performance,  an  organization  needs  to  measure 
both  its  internal  processes  and  its  final  outputs  and  outcomes.  However,  a  word  of  caution,  payouts 
from  a  PGS  plan  should  only  be  based  on  improvements  in  results.  Processes  should  be  measured, 
and  efforts  should  be  made  to  continuously  improve  processes,  but  payouts  should  only  be  made 
on  improvements  in  results.  This  statement  is  true  for  two  reasons.  First,  an  organization  can 
improve  processes  without  necessarily  improving  the  output  or  outcome.  For  example,  an 
organization  could  change  its  training  programs  so  that  the  skill  level  of  its  employees  is  improved, 
but  if  the  training  were  in  areas  that  did  not  contribute  to  improved  productivity  (e.g.,  seminars  on 
how  to  prepare  for  retirement),  it  would  not  make  sense  to  tie  these  improvements  to  a  productivity 
gain.  Second,  linking  payouts  directly  to  process  improvements  would  constitute  a  form  of  double 
counting.  Improvements  in  many  processes  ultimately  show  up  as  improvements  in  the  final  output 
or  outcome.  If  some  portion  of  the  payout  is  based  on  process  improvements,  and  some  other 
portion  is  based  on  improvements  in  the  end  product  that  resulted  from  the  process,  the 
organization  is  essentially  paying  twice  for  the  same  improvement. 

How  to  Use  This  Document 

As  noted  earlier.  Table  1  shows  the  major  components  of  organizational  effectiveness.  Table  1 
provides  examples  of  measures  that  may  be  useful  in  PGS.  Within  each  of  the  four  broad  areas 
shown  in  Table  1  (productivity,  financial  performance,  stakeholder  relations,  and  resource 
development).  The  measures  provided  cover  the  full  range  of  measures  discussed  earlier  (i.e., 
input,  process,  output,  outcome).  When  and  how  these  measures  should  be  used  will  be  addressed 
next.  However,  remember  that  different  organizations  have  different  measurement  requirements. 
Because  of  this  heterogeneity,  the  measures  discussed  are  general  in  nature,  but  specific  enough  to 
give  most  organizations  a  good  idea  of  how  to  generate  their  own  set  of  measures. 

Measurement  requirements  are  determined  by  the  specific  key  result  areas  defined  by  the 
organization.  In  general,  all  organizations  will  have  key  result  areas  that  reflect  the  quantity, 
quality,  and  timeliness  of  their  products  and  services,  but  their  importance  will  vary  across 
organizations.  For  example,  a  supply  center  may  emphasize  timeliness  of  deliveries,  while  a  data 
processing  center  stresses  accuracy  of  completed  jobs.  Both  organizations  have  timeliness  and 
accuracy  as  key  result  areas,  but  the  emphasis  and  weightings  differ.  Each  organization  will  select 
and  develop  some  performance  measures  unique  to  their  requirements  and  areas  of  emphasis,  but 
selection  and  development  of  measures  will  likely  overlap  (e.g.,  both  will  want  indicators  of 
customer  satisfaction,  both  will  probably  want  to  measure  the  volume  of  their  work).  Therefore, 
how  any  given  organization  uses  this  document  will  depend  on  the  unique  character  of  the 
organization  and  the  distinctive  needs  of  that  organization. 

We  recommend  using  this  document  as  a  guide  to  selecting  and  developing  a  set  of  measures 
best  suited  to  that  organization.  The  organization  probably  will  want  some  measures  from  each  of 
the  four  major  areas  in  Table  1,  but  from  the  standpoint  of  PGS  many  of  the  measures  will  come 
from  the  first  two  areas  (productivity  and  financial  performance).  We  give  many  examples  of 
measures  for  white  collar  organizations  in  this  document,  and  many  of  these  can  be  adopted  by 
almost  any  organization  without  much  modification.  However,  because  every  organization  is 
unique,  each  organization  must  develop  performance  measures  that  reflect  its  distinctive 
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that  our  earlier  discussion  of  measurement  issues  and  the  numerous  examples  to  follow  will  clarify 
and  simplify  this  task.  The  important  thing  to  keep  in  mind  when  developing  performance 
measures  for  a  PGS  system  is  “improvement.”  You  want  your  measures  to  show  whether  you  have 
made  improvements  in  your  key  result  areas.  You  want  measures  with  numbers  that  tell  you 
whether  you  have  done  better  or  worse. 


Productivity 

Productivity  refers  to  a  statistical  measure  that  reflects  how  inputs  to  the  organization  (e.g., 
labor,  capital,  and  equipment)  are  used  to  produce  outputs  (goods  and  services).  The  terminology 
in  the  area  of  productivity  is  confusing  and  not  always  used  consistently.  Terms  such  as  efficiency, 
performance,  effectiveness,  and  quality  are  sometimes  used  interchangeably  with  productivity  (see 
Tatum,  Shaw,  &  Main,  1996,  for  a  more  complete  discussion  of  definitions  of,  and  relationships 
between,  various  terms).  Here,  productivity  is  expressed  as  a  ratio  of  inputs  to  quality  outputs  (see 
Tatum,  Shaw,  &  Main,  1996),  and  measures  of  productivity  fall  into  two  general  areas;  process 
quality  (i.e.,  efficiency)  and  output  quality. 

Process  Quality  (Efficiency) 

The  traditional  view  of  an  organization  is  vertical  in  that  each  department  or  business  unit 
functions  on  its  own  with  its  own  management  hierarchy  (Rumor  &  Brace,  1991).  A  more  useful 
approach  is  a  horizontal,  or  “systems,”  view  that  cuts  across  functional  boundaries  and  is  organized 
around  processes  that  include  inputs  to  the  system,  the  transformation  of  these  inputs  into  an  output 
(product  or  service),  and  the  delivery  of  the  output  to  a  customer.  To  achieve  the  greatest  efficiency 
in  transforming  the  inputs  into  the  outputs,  the  organization  must  be  able  to  measure  quantities  of 
inputs  and  outputs,  and  must  be  able  to  measure  improvement  efforts  in  the  process  of  transforming 
the  inputs  into  outputs.  Efficiency  is  usually  expressed  as  an  input/output  ratio.  What  follows  are 
some  alternative  ways  of  measuring  the  quantities  of  the  inputs  and  the  outputs  that  go  into  this 
ratio. 

Input  Quantity 

Inputs  are  the  resources  used  in  the  production  of  outputs.  The  four  resources  most  commonly 
considered  as  important  are:  (1)  labor  (direct  and  indirect),  (2)  capital  (equipment  and  facilities), 
(3)  materials  (raw  material,  purchased  parts,  and  supplies),  and  (4)  energy  (fuel  and  power).  These 
resources  are  also  described  by  Nebeker,  Neuberger,  and  Hulton  (1983).  Most  of  the  following 
examples  of  measures  of  input  quantity  are  expressed  in  nonfinancial  terms  (the  financial 
equivalents  are  shown  in  the  section  on  financial  performance).  These  measures  could,  in  principle, 
be  expressed  as  dollar  values,  but  only  if  the  organization  can  be  sure  that  these  are  constant 
dollars.  If  the  dollars  values  do  not  remain  stable  (e.g.,  the  cost  of  labor  and  materials  fluctuates), 
then  it  is  not  a  good  idea  to  express  these  inputs  as  dollar  values.  Instead  we  recommend  the 
nonfinancial  quantities  listed  here. 

1.  Total  Labor  Hours  Worked:  A  measure  of  the  number  of  labor  hours  (both  direct  and 
indirect)  used  to  produce  the  products  and  services.  For  example,  in  a  data  processing  operation, 
this  would  be  the  total  labor  hours  used  for  conducting  training,  writing  software,  running  the 
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computers,  and  all  support  functions  (e.g.,  secretarial  services,  purchasing,  travel).  In  a  research 
and  development  lab,  this  would  be  a  measure  of  the  scientist  and  technicians  used  on  research 
projects  as  well  as  the  support  personnel.  You  show  improvement  in  this  measure  by  reducing  the 
labor  hours  used  to  produce  the  same  quantity  and  quality  of  output  as  in  the  past  (measures  of 
output  quantity  and  quality  are  discussed  in  a  later  section). 

2.  Capital  Assets:  The  number  of  major  capital  investments  made.  This  measure  can  be 
expressed  as  the  cost  or  the  depreciated  value  of  equipment  or  facilities.  In  general,  improvements 
in  this  measure  are  shown  by  increasing  output  for  a  given  level  of  capital  investments  or 
decreasing  capital  costs  without  having  a  negative  effect  on  the  product  or  service  and  without 
jeopardizing  long  term  survival.  If  it  can  be  shown  that  investing  in  capital  assets  improves  the 
product  or  service  and  enhances  survival,  then  more,  rather  than  less,  investment  is  good. 

3.  Standard  Units  of  Material  Used:  Items  such  as  the  number  of  units  of  material  used, 
pounds  of  material  used,  or  number  of  feet  used.  These  same  metrics  can  also  be  used  to  measure 
waste  and  scrap.  An  organization  can  show  improvement  by  lowering  the  amount  of  material  used 
to  produce  the  same  quantity  and  quality  of  output. 

4.  Units  of  Energy  Consumed:  The  amount  of  energy  used  in  the  production  process. 
Examples  of  this  are  Kilowatt  hours  and  Therms.  Information  from  these  measures  can  be 
compared  to  past  usage  and  also  to  the  usage  of  other  organizations.  Improvements  show  up  as 
lower  units  of  energy  consumed  relative  to  past  usage  or  relative  to  other  similar  organizations. 

Output  Quantity 

An  organization’s  internal  operation  results  in  some  output  (a  product  or  service).  Output 
quantity  refers  to  the  number  of  rmits  completed  in  some  time  period  (Nebeker  et  al.,  1983). 
Measuring  output  units  has  typically  been  a  challenge  for  white  collar  organizations.  Identifying  and 
qualifying  specific  output  units  is  often  difficult.  According  to  Ringham  (1982),  outputs  can  be 
categorized  as  product  units  (pieces,  dozens,  or  cases),  physical  units  (pounds,  square  feet,  gallons, 
or  bushels),  sales/service  units  (orders,  deliveries,  or  invoices),  value  (sales  value  or  value  added), 
and  work  content  (standard  labor  hours,  standard  machine  hours,  or  estimated  labor  and  machine 
hours).  Although  these  categories  cover  the  outputs  in  a  general  sense,  this  classification  is  not  too 
helpful  when  an  organization  (especially  a  white  collar  organization)  is  trying  to  be  specific  about 
identifying  and  quantifying  its  outputs.  The  best  advice  that  can  be  given  to  any  organization  is  to 
let  customers  define  outputs.  After  customers  have  specified  a  specific  product  or  service  they  need, 
then  investigate  ways  to  measure  the  quantity  of  this  output.  Guidelines  provided  by  the  Bureau  of 
Labor  Statistics  (BLS)  can  be  helpful  in  quantifying  outputs.  The  BLS  publishes  an  annual  report 
from  its  Federal  Productivity  Measurement  System  (FPMS)  that  summarizes  labor  productivity 
trends  for  28  broad  government  functions,  including  many  DOD  functions  (see  Bureau  of  Labor 
Statistics,  1993c;  Bureau  of  Labor  Statistics,  1993d).  The  BLS  has  several  specific  guidelines  for 
the  kind  of  output  data  reported  by  the  agencies  that  are  included  in  the  FPMS  (Forte,  1993;  Bureau 
of  Labor  Statistics,  1993a).  These  guidelines  include: 

•  Output  measures  should  reflect  the  final  products  and  services  of  an  organization.  Final 
products  and  services  are  outputs  that  are  consumed  by  customers  or  agencies  outside  the 
boundaries  of  the  organization.  For  example,  a  library  purchases  books  and  periodicals,  catalogs 
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these  materials,  and  then  lends  them  to  individuals  and  other  institutions.  The  outputs  are  the  books 
and  periodicals  lent.  The  purchasing  and  cataloging  activities  are  intermediate  tasks,  and  would  not 
be  counted  as  outputs. 

•  If  sets  of  output  measures  are  not  homogeneous  (i.e.,  they  are  not  of  equivalent  complexity 
or  they  do  not  have  the  same  labor  requirements),  then  a  weighting  factor  should  be  applied  to  the 
measures.  For  example,  if  a  printing  shop  measures  pages  of  documentation,  they  should 
distinguish  between  long  run  pages  and  short  run  pages  by  assigning  different  values  to  the 
measures. 

•  To  the  extent  possible,  output  measures  should  reflect  repetitive  processes  (e.g.,  periodic 
audits,  frequent  requests,  regularly  scheduled  maintenance)  rather  than  processes  that  occur 
infrequently  or  on  a  one-time  basis.  Repetitive  processes  can  be  checked  for  consistency  and 
stability,  whereas  nonrepetitive  processes  are  subject  to  unexpected  variation  and  instability. 

•  Measures  should  reflect  outputs  that  are  related  to  the  workloads  of  the  organizations  and  not 
work  performed  by  third  parties  or  outsourcing  agencies.  If  an  organization  contracts  out  its 
janitorial  services,  for  example,  it  can  measure  the  processing  of  the  contract  (work  that  the 
organization  performs),  but  should  not  count  the  custodial  work  performed  by  the  contractor  as  part 
of  the  organizations’s  outputs. 

•  If  work  is  the  joint  effort  of  two  or  more  organizations,  the  output  measures  should  reflect  the 
separate  contributions  of  each  organization.  For  example,  if  several  regional  offices  contribute  data 
to  a  headquarters  command  that  then  produces  a  report,  the  data  supplied  by  a  particular  office 
would  be  the  measure  of  output  from  that  office.  The  final  report  can  be  counted  as  an  output  for 
the  headquarters,  provided  that  headquarters  is  a  separate  organizational  entity  from  the  regional 
offices.  If  the  regional  offices  and  headquarters  are  all  part  of  the  same  organizational  entity,  then 
it  would  not  be  proper  to  count  the  data  and  the  report  as  separate  outputs.  In  this  case,  the  data  are 
intermediate  steps  and  the  final  report  is  the  output.  Counting  the  intermediate  steps  and  the  final 
report  as  outputs  is  equivalent  to  double  counting. 

•  For  government  organizations  that  work  on  a  fiscal  year  basis,  performance  measures  should 
be  scaled  for  the  fiscal  year.  For  example,  if  it  takes  5  years  to  build  a  ship,  it  would  be  improper 
to  count  one  ship  built  in  the  fifth  year  and  zero  in  each  of  the  first  4  years.  Instead,  it  would  be 
better  to  measure  the  portion  completed  in  each  year,  or  to  break  the  ship  building  process  into 
major  steps  and  measure  the  completion  of  steps  in  each  year. 

Because  outputs  vary  tremendously  across  organizations,  only  some  general  examples  of  how 
to  measure  output  quantity  can  be  offered,  based  on  Ringham’s  (1982)  classification.  (See  the 
report  by  the  Bureau  of  Labor  Statistics,  1993b,  for  a  more  extensive  list  of  possible  output 
measures.)  As  with  the  input  measures,  most  of  the  following  examples  of  measures  of  output 
quantity  are  expressed  in  nonfinancial  terms  (the  financial  equivalents  are  shown  in  the  section  on 
financial  performance).  Again,  these  measures  could,  in  principle,  be  expressed  as  dollar  values, 
but  only  if  the  organization  can  be  sure  that  these  are  constant  dollars.  If  the  dollars  values  do  not 
remain  stable  (e.g.,  customers  renegotiate  lower  prices),  then  it  is  not  a  good  idea  to  express  these 
outputs  as  dollar  values  and  we  recommend  the  following  nonfinancial  quantities  listed  here.  The 
BLS  also  discourages  using  dollars  as  an  output  measure  (Forte,  1993). 
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1.  Raw  Units  of  Goods:  A  count  of  the  total  number  of  goods  produced.  For  example,  an  audit 
service  can  count  the  number  of  audit  reports  produced.  A  research  and  development  center  can 
count  pages  of  documentation,  or  the  number  of  patents  granted.  Of  course,  counting  these  outputs 
is  far  more  complex  than  implied  here  (e.g.,  you  may  have  to  weight  each  output  by  its  level  of 
complexity,  or  you  may  have  to  take  into  consideration  changes  in  the  mix  of  work  over  time). 
Once  the  organization  develops  a  standardized  way  of  counting  the  outputs,  improvement  is  then 
reflected  in  an  increase  in  the  number  of  raw  units  produced  given  a  constant  amount  of  input  and 
no  decline  in  quality  (measures  of  quality  are  discussed  in  a  later  section). 

2.  Raw  Units  of  Services:  Similar  to  raw  units  of  goods  except  this  is  a  count  of  the  total 
number  of  services  delivered.  As  with  raw  goods,  complexity  may  be  an  issue  (some  services  may 
be  more  complex  than  others)  and  so  a  weighting  scheme  may  be  required.  For  example,  a 
contracting  office  can  count  the  number  of  completed  contracts  issued,  but  each  contract  is  scaled 
in  terms  of  complexity  so  that  more  credit  (or  weight)  is  given  for  some  contracts  (see  Forte,  1993; 
Tatum  &  Nebeker,  1996  for  examples  of  how  to  weight  indicators).  Another  example  might  be  a 
training  center  that  counts  the  number  of  recruits  trained.  If  all  recruits  get  the  same  training  then 
an  unweighted  count  is  appropriate.  If  different  levels  of  training  are  delivered,  then  a  weighting 
scheme  reflecting  the  difficulty  of  the  training  should  be  employed.  In  both  examples,  if  an 
increase  in  these  quantities  (contracts,  recruits)  is  achieved  without  increasing  the  inputs  or 
lowering  the  quality,  improvement  has  occurred. 

5.  Sales  Value:  Although  as  a  general  rule  it  is  not  a  good  idea  to  use  dollars  as  an  output 
measure,  there  are  cases  in  which  the  revenue  generated  from  an  operation  can  be  a  good  indicator 
of  output.  For  example,  a  commissary  can  calculate  the  deflated  dollar  value  of  the  sale  of  food  and 
other  items.  If  the  mix  of  items  varies  widely  in  terms  of  dollar  value  (e.g.,  million  dollar  tanks  vs. 
much  less  expensive  rifles),  then  sales  value  probably  is  not  a  good  output  measure,  and  an 
alternative  should  be  found.  However,  assuming  a  homogeneously  priced  set  of  items,  a  continual 
increase  in  the  revenue  is  an  indication  of  improvement. 

4.  Physical  Measurement:  Assess  the  physical  changes  in  a  product  such  as  weight  or  volume. 
For  example,  a  supply  center  measures  the  volume  changes  or  weight  changes  in  its  shipments.The 
ability  to  move  a  greater  volume  of  shipments  for  a  constant  or  lower  resource  use  suggests 
improved  performance.  One  caution  here  has  to  do  with  the  mix  of  work  from  one  period  to  the 
next.  If  the  organization  deals  with  a  variety  of  outputs,  and  the  mix  of  these  outputs  changes 
frequently,  there  may  be  a  problem  making  comparisons  from  one  performance  period  to  the  next 
Organizations  in  this  situation  should  insure  that  the  outputs  are  expressed  in  equivalent  units,  or 
should  measure  over  extended  periods  so  that  output  mix  balances  out  over  time. 

5.  Work  Content:  Measure  the  effort  that  goes  into  a  particular  job  or  task.  For  example,  a 
public  works  center  may  have  a  set  of  standards  for  how  long  a  job  should  take.  A  measure  of 
output  can  be  the  ratio  of  standard  time  for  completed  work  to  the  actual  time  to  accomplish  the 
job.  As  this  ratio  improves,  organizational  performance  is  increased. 

Process  Improvement 

The  imperative  of  every  organization  should  be  continuous  improvement  of  the  quality  and 
efficiency  of  each  process  within  that  organization  (see  Nebeker,  Wolosin  &  Tatum,  1996,  for  a 
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detailed  model  of  organizational  improvement).  Process  improvement  begins  with  knowing  your 
customers  requirements,  and  then  working  backward  from  that  point  and  making  improvements  to 
each  process  and  subprocess  that  contributes  to  meeting  these  customer  requirements.  Many  tools 
and  techniques  support  for  process  improvement.  Many  of  these  improvement  tools  yield  measures 
of  how  much  the  organization  has  improved  over  time.  These  measures  are  diagnostic  in  that  they 
help  the  organization  correct  current  problems  and  prevent  future  mishaps.  A  few  of  these  tools 
and  their  corresponding  measures  are  listed  below. 

1.  Data  From  Control  Charts:  This  approach  to  process  improvement  generates  a  statistical 
measure  of  the  amount  of  variability  or  stability  in  an  organization’s  processes  over  time.  For 
example,  one  of  the  critical  attributes  of  a  technical  document  is  the  number  of  typographical 
errors.  A  control  chart  can  show  the  average  number  of  errors  and  the  distribution  of  those  errors 
over  time.  As  an  organization  improves  its  document  processing  procedures,  the  control  chart  will 
reflect  these  improvements  in  a  number  of  ways:  (a)  fewer  “out-of-control”  points  (special  causes 
of  variation)  will  occur,  (b)  the  overall  variability  in  the  distribution  of  errors  will  decrease,  and  (c) 
the  average  number  of  errors  will  decrease  and  move  closer  to  the  ideal  zero  point.  For  more 
information  on  the  use  of  control  charts  see  Deming  (1986),  Ishikawa  (1985),  Rumor  and  Brace 
(1991),  Wheeler  and  Chambers  (1992),  or  Gitlow,  Gitlow,  Oppenheim,  and  Oppenheim  (1989). 

2.  Benchmarking:  This  technique  systematically  compares  performance  of  a  process  to  a 
similar  process  for  another  organization  with  an  excellent  reputation.  As  with  control  charts  and 
other  process  measures,  benchmarking  is  a  good  diagnostic  tool  (it  shows  the  organization  where 
the  problems  lie),  but  does  not  directly  suggest  exactly  how  the  problem  can  be  solved.  The 
creative  talent  in  the  organization  must  devise  changes  that  will  improve  the  process.  Once  these 
changes  are  implemented,  data  from  benchmarking  tests  can  be  used  to  determine  if  the  changes 
led  to  improvements.  Improvements  are  revealed  by  forming  a  ratio  of  the  organization’s 
performance  to  the  performance  of  the  benchmark.  As  this  ratio  grows  larger,  the  organization  is 
demonstrating  improvement.  Measures  from  this  approach  also  yield  information  on  the  best 
practices  available  for  making  improvement. 

3.  Time  on  Inspection:  The  time  spent  inspecting  the  final  product  or  service  can  be  recorded. 
As  the  internal  processes  of  the  organization  improve  fewer  and  fewer  labor  hours  should  be 
devoted  to  inspecting  the  final  output.  This  information  can  also  be  used  to  tell  the  organization 
how  much  cost  could  be  eliminated  by  building  quality  up  front. 

4.  Time  on  Rework:  The  time  spent  on  rework  can  be  recorded  and  compared  to  past  years  or 
to  other  organizations.  Process  improvements  show  up  as  decreases  in  labor  hours  spent  on  rework. 
The  cost  of  rework  can  be  combined  with  the  cost  of  inspection  to  provide  an  indication  of  how 
much  money  could  be  saved  if  quality  were  built  into  the  product  or  service  from  the  beginning. 

5.  Amount  of  Scrap  and  Waste:  The  amount  and  costs  of  scrap  and  waste  can  be  recorded. 
Process  improvements  are  revealed  by  lower  amounts  of  scrap  and  waste.  This  information  can 
provide  the  organization  with  an  idea  of  how  much  the  yield  could  be  increased  from  prevention 
or  self-inspection. 
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Output  Quality 

Besides  measuring  efficiency  (the  quality  of  the  process),  an  organization  needs  to  measure  the 
quality  of  the  end  products  and  services.  This  measurement  forms  an  important  dimension  of 
productivity  measurement.  Quality  is  the  degree  to  which  each  unit  of  output  possesses  desirable 
characteristics  (Nebeker  et.,  1983).  Quality  ultimately  is  based  on  the  customers  reaction  to  the 
product  or  service.  If  the  customer  is  satisfied,  then  the  product  or  service  has  desirable 
characteristics. 

End-Product  Inspection 

Although  end-product  inspection  cannot  create  quality  products  and  services,  it  can  uncover 
and  correct  defects  and  prevent  them  from  reaching  the  customer.  Of  course,  in  cases  of  financial 
transactions,  safety-related  work,  and  product  liability,  end-product  inspections  may  be  required 
(Deming,  1986;  Ishikawa,  1985).  The  problem  with  end-product  inspection  as  a  quality  control 
method  is  that,  when  defects  are  found,  the  only  corrective  actions  available  are  reworking  the 
product  or  consigning  it  to  scrap.  In  either  case,  productivity  suffers  and  costs  rise.  The  yield  is 
lower  due  to  the  extra  labor  involved  in  the  rework  and  the  cost  of  the  material  which  has  to  be 
scrapped.  In  addition,  products  that  have  been  adjusted  or  reworked  are  more  likely  to  break  down, 
which  is  exactly  the  opposite  of  quality  assurance  (Ishikawa,  1985). 

Self-inspection  by  each  worker  in  the  process  is  preferred  to  end-product  inspection. 
Self-inspection  moves  the  quality  assurance  farther  upstream  and  allows  corrective  and  preventive 
actions  to  take  place  at  an  earlier,  less  costly,  stage  of  the  process.  Self-inspection  makes  people 
more  aware  of  problems  and  leads  to  earlier  identification  and  corrective  action.  If  an  organization 
has  not  developed  a  mature  system  of  self-inspection  and  prevention,  and  must  rely  on  end-product 
inspection,  there  is  really  only  one  measure  of  output  quality: 

1.  Proportion  of  Output  Units  Passing  Inspection:  This  measure  is  obtained  by  counting  the 
number  of  units  that  pass  inspection  and  get  into  the  hands  of  the  customer.  Improvements  in  the 
quality  of  the  product  or  service  are  shown  by  increases  in  the  proportion  of  outputs  that  pass 
inspection. 

Product/Service  Attributes 

Product/service  attributes  are  those  properties  of  the  output  that  the  consumer  or  end-u^r 
determines  to  be  of  value.  Some  of  these  are  meeting  customer  desires  (e.g.,  an  end-user  may  desire 
certain  extra  features  such  as  a  MS  Windows  interface  for  a  software  application),  uniformity  (e.g., 
a  sponsor  may  want  research  reports  written  in  a  standardize  form),  fitness  for  use  (e.g.,  a  customer 
may  require  travel  orders  to  conform  to  a  certain  format  so  they  can  input  the  information  into  their 
computers),  and  reliability  (e.g.,  a  consumer  may  demand  error-free  data  processing  of  its  mailing 
list).  The  following  list  provides  alternative  measures  of  output  quality  that  fall  into  the  above 
categories.  It  is  important  to  keep  in  mind  that  these  are  not  mutually  exclusive  measures.  Ideally, 
the  organization  should  use  as  many  of  these  as  feasible  so  that  quality  is  measured  in  a  variety  of 
ways. 
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1.  Customer  Surveys:  Customers  can  be  surveyed  to  ask  about  their  level  of  satisfaction  as 
well  as  current  and  future  needs  or  desires.  Possible  survey  items  include:  Rate  the  level  of 
satisfaction  for  the  following  products/services.  To  what  extent  do  our  products  or  services  meet 
your  current  needs?  To  what  extent  is  our  product  or  service  delivered  on  time?  To  what  extent  is 
it  delivered  in  the  right  amounts?  Rate  the  reliability  of  product/service.  To  what  extent  are  we 
helpful  when  you  call  us  with  a  problems?  An  example  of  the  use  of  survey  data  might  come  from 
a  training  command.  The  command  can  survey  the  trainees  upon  completion  of  the  courses.  These 
surveys  can  be  followed  by  additional  surveys  of  trainees  and  their  supervisors  several  months  later 
to  see  if  the  training  continues  to  be  useful.  These  data  can  then  be  used  to  assess  whether  the 
training  command  is  making  improvements  in  the  course  materials,  instruction,  and  offerings. 

2.  Statistical  Process  Control  (SPC)  Tools:  There  are  several  tools  used  in  SPC  that  provide 
valuable  data  on  product  and  service  attributes.  For  example,  control  charts  can  be  used  to  show 
the  uniformity  of  a  product  or  service  and  whether  the  average  value  is  close  to  the  ideal  (nominal) 
value  of  the  output  (see  Gitlow  et  al.,  1989,  chapters  8-11).  Similarly,  the  Taguchi  Loss  Function 
can  also  show  output  uniformity  and  conformance  to  the  customer  ideal  (see  Gitlow  et  al.,  1989, 
chapter  15).  Finally,  process  capability  (see  Gitlow  et  al.,  1989,  chapter  14)  is  a  tool  that  can  be 
used  to  show  how  well  the  output  conforms  to  the  specification  limits.  If  the  data  from  these  SPC 
tools  shows  that,  over  time,  the  product  or  service  is  more  uniform  (less  variable)  and  conforms  to 
requirements,  this  is  a  good  indication  of  improved  quality.  For  example,  a  contracting 
organization  records  the  time  required  to  issue  a  contract  and  implements  changes  in  the  hope  of 
improving  contract  deliveries.  The  above  SPC  tools  can  reveal  reductions  in  the  variability  of  issue 
time,  as  well  as  show  how  closely  the  issue  time  conforms  to  the  time  requirements  of  the 
customer. 

5.  Structured  Interviews:  Personal  interviews  (either  direct  or  by  phone)  can  be  conducted 
with  customers  using  the  same  questions  you  would  pose  in  a  survey.  Interviews  can  provide 
information  of  greater  depth  than  surveys  because  customer  comments  can  be  explored 
immediately.  For  example,  external  customers  can  be  asked  the  extent  to  which  the  product  or 
service  is  meeting  their  needs  and  preferences,  and  then  dig  deeper  into  problem  areas  revealed  by 
the  customers  spontaneous  comments.  The  results  from  these  interviews  can  be  quantified  (see 
Dooley ‘s,  1984.  pp.  89-91,  discussion  of  content  analysis  and  paralinguistic  process  analysis  for 
guidance)  and  an  index  of  customer  approval  and  problem  identification  can  be  developed.  As  the 
approval  ratings  increase,  or  as  the  problem  areas  decrease,  product/service  quality  shows 
improvement. 

4.  Customer  Complaints:  An  organization  can  measure  how  well  it  is  satisfying  its  customers 
by  the  number  and  type  of  complaints  that  are  registered.  These  data  will  reflect  improvement  over 
previous  years  if  there  is  a  decrease  in  the  number  and  severity  of  complaints. 

5.  Unsolicited  Letters  of  Appreciation:  This  is  another  way  that  the  organization  can  measure 
how  well  it  is  doing  in  satisfying  its  customers.  The  data  can  then  be  compared  to  previous  years 
to  determine  if  the  product  or  service  is  improving.  The  organization  can  also  compare  the  data  to 
the  number  and  type  of  complaints  in  order  to  see  the  areas  in  which  performance  is  good  and  the 
areas  in  need  of  improvement. 
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6.  Returns:  Failures  of  a  product  can  be  measured  by  the  number  of  items  that  are  returned  for 
a  refund  or  repair,  and  the  data  can  be  compared  to  data  from  previous  time  periods.  When  products 
fail,  the  mean-time-between-failure  is  a  statistic  that  is  commonly  used  to  indicate  improvement 
(decreases  in  mean-time-between-failure  are  indicative  of  improvement). 

7.  Comment  Cards:  Failures  in  service  can  be  assessed  by  issuing  customer  response  cards. 
For  example,  as  tvith  commercial  hotels,  a  Bachelor  Officer’s  Quarters  can  include  customer 
feedback  cards  in  the  room  that  solicit  comments  on  what  was  done  well  and  what  needs 
improvement.  An  index  of  overall  improvement  of  service  might  be  the  ratio  of  positive  to  negative 
comments.  As  another  example,  an  activity  that  makes  travel  arrangements  can  include  a  response 
card  with  every  set  of  travel  orders,  and  the  ratio  of  positive  to  negative  comments  will  reflect 
improvements  in  the  service. 

Timeliness 

Timeliness  involves  the  time  criticality  of  delivery  (e.g.,  supplies  are  delivered  when  needed), 
the  shelf  life  of  inventory  (e.g.,  pharmaceuticals  are  used  before  their  effective  date  expires),  and 
meeting  milestones  and  deadlines  (e.g.,  delivering  a  product  on  an  agreed  upon  schedule). 

L  Just-in-Time  Index:  In  many  organizations,  the  ideal  is  to  have  information  or  materials 
delivered  to  the  customer  “just-in-time.”  For  example,  an  organization  is  delivering  training  to 
another  organization,  it  is  critical  that  the  training  be  conducted  at  the  time  it  is  needed  and  not 
months  too  early  or  too  late  to  be  used  effectively.  A  Just-in-Time  index  is  a  measure  of  how  close 
in  time  the  product  or  service  is  to  the  target  time.  The  closer  the  actual  delivery  time  is  to  the  target 
time,  the  better  the  performance  of  the  organization,  and  improvements  are  shown  by  ever-smaller 
differences  in  time. 

2.  Shelf  Life:  This  is  a  measure  of  the  life  of  a  product  before  it  is  used.  The  shorter  the  shelf 
life  of  a  product,  the  more  frequently  it  must  be  reordered  and  restocked.  Organizations  for  which 
shelf  life  is  an  important  concern  are  hospitals  (where  the  useful  life  of  the  drugs  is  a  critical  issue) 
and  warehouses  (where  the  shelf  life  of  the  supplies  is  of  paramount  concern). 

3.  Conformance  to  Schedule:  Records  can  be  kept  on  how  well  the  organization  is  doing  at 
delivering  the  right  product  or  service  at  the  right  time  in  the  right  amount.  As  an  example,  a 
research  and  development  organization  may  have  deadlines  for  when  summary  reports  are  due  to 
the  sponsor.  Good  performance  would  be  reflected  by  meeting  the  deadline  or,  under  some 
circumstances,  delivering  ahead  of  schedule. 

Financial  Performance 

The  second  area  in  Table  1  shows  financial  performance.  Financial  performance  refers  to  the 
profitability  (in  the  private  sector),  budgeting  (in  the  public  sector),  and  market  share  of  an 
organization.  Effective  organizations  are  able  to  compete  successfully  and  acquire  scarce  resources 
for  manufacturing  their  goods  or  providing  their  services.  In  turn,  these  resources  are  then 
transformed  in  a  manner  that  is  financially  beneficial  to  the  organization.  This  financial  benefit  is 
expressed  as  a  ratio  between  the  revenue  generated  and  the  costs  to  produce  the  product  or  service. 
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Budgeting 

Most  organizations  in  the  private  sector  are  concerned  with  the  maximization  of  profits  as  an 
objective.  By  contrast,  most  organizations  in  the  public  sector  strive  to  stay  within  budget.  Ideally, 
when  an  organization  chooses  among  alternative  business  strategies,  it  should  be  able  to  forecast 
all  future  revenues  and  costs  accurately  and  can  achieve  some  degree  of  control  over  both. 
However,  in  the  public  sector  improved  budgeting  normally  means  controlling  and  reducing  costs 
relative  to  a  mostly  fixed  source  of  revenue.  Success  is  measured  in  terms  of  improving  the  ratio 
of  revenue  to  cost.  What  follows  are  alternative  ways  of  measuring  the  revenues  and  costs  that  form 
this  ratio. 

Revenue 

Organizations  attempting  to  project  their  budget  must  consider  the  revenue  generated  from  the 
product  or  service.  Although  some  government  organizations  operate  on  a  market-oriented, 
revenue  generating  basis  (e.g..  Defense  Business  Operating  Fund),  many  others  operate  on 
appropriated  funding  in  which  the  revenues  are  fixed  by  higher  authority  on  a  yearly  basis.  A  data 
processing  operation,  for  example,  represents  a  somewhat  market-oriented  government  enterprise. 
A  data  processing  operation  would  look  at  (1)  the  cost  to  provide  the  services  it  offers,  (2)  what  the 
customer  might  have  to  pay  elsewhere,  and  (3)  the  “profit”  margin  it  can  reasonably  expect.  From 
these  considerations,  the  data  processing  operation  determines  the  rates  to  charge  the  customer 
which  translates  into  potential  revenue.  A  DOD  headquarters  operation,  on  the  other  hand,  is  a 
more  traditional  government  institution.  Headquarters  usually  receive  appropriated  funds  from  a 
Congressionally-approved  budget.  These  funds  are  obligated  on  an  annual  basis  and  the 
headquarters  must  pay  its  salaries,  purchase  its  materials,  issue  contracts,  and  support  its  field 
activities  from  these  fixed  dollar  amounts.  Regardless  of  whether  the  organization  is  market- 
oriented  or  traditionally  funded,  total  revenue  generated  constitutes  the  primary  financial  measure. 

1.  Total  Revenue:  Measures  the  total  dollars  generated  from  the  sale  of  all  goods  and  services 
sold  to  customers,  or  from  funds  received  from  various  funding  agencies.  In  general,  the  more 
revenue  generated  the  better  the  financial  performance.  Successful  financial  performance  means 
acquiring  sufficient  revenue  to  cover  operating  costs  and  to  allow  a  certain  degree  of  future 
expansion. 

Costs 

The  number  of  input  units  purchased  (e.g.,  labor,  capital,  materials,  and  energy)  multiplied  by 
the  price  paid  for  each  unit  yields  the  total  cost  of  running  an  organization.  The  price  of  labor, 
capital,  material,  or  energy  depends  on  the  supply  and  demand  of  the  resources.  The  prudent 
organization  seeks  to  purchase  the  highest  quality  units  at  the  lowest  possible  price.  The  following 
list  represents  the  major  cost  factors  in  a  typical  organization.  This  list  bears  a  remarkable 
resemblance  to  the  list  of  inputs  quantities  described  under  productivity.  The  present  list  represents 
the  dollar  equivalents  of  those  input  units. 

1.  Dollars  Spent  on  Total  Labor:  A  measure  of  the  dollar  costs  to  purchase  the  labor  hours  used 
to  support  the  entire  organization  (both  direct  and  indirect  labor). 
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2.  Depreciation  of  Capital  Assets:  The  estimated  cost  of  using  property,  plant,  and  equipment. 
For  example,  assume  a  machine  is  purchased  for  $100,000  and  then  sold  8  years  later  for  $20,000. 
The  cost  of  using  this  fixed  asset  is  $80,000  over  its  8-year  useful  life.  In  any  given  period,  the  cost 
is  a  prorated  share  of  the  $80,000. 

3.  Dollars  Spent  on  Materials:  The  dollar  cost  of  the  materials  purchased  by  categories  (e.  g., 
supplies,  minor  equipment). 

4.  Dollars  Spent  on  Energy  Consumed:  A  measure  of  the  dollar  cost  of  purchasing  the  energy 
used  in  the  manufacturing  of  goods  or  the  delivery  of  services. 

Once  the  revenues  and  costs  are  measured,  the  financial  success  can  be  assessed  by  examining 
the  ratio  of  revenue  to  cost.  As  the  ratio  increases,  the  financial  performance  of  the  organization 
improves.  It  is  important  to  note,  however,  that  financial  performance  is  not  necessarily  the  same 
as  productivity.  A  high  revenue/cost  ratio  does  not  necessarily  imply  that  the  organization  is  more 
productive  because  this  ratio  can  be  influenced  by  factors  that  are  independent  of  the  output/input 
productivity  ratio  (e.g.,  price  increases  because  of  higher  demand  or  lower  supply  can  increase  the 
revenue  more  than  it  increases  costs).  Likewise,  a  lower  revenue/cost  ratio  does  not  necessarily 
imply  that  the  organization  is  less  productive  for  much  the  same  reason  (e.g.,  if  the  customer 
negotiated  a  lower  price  for  a  product,  or  competition  drove  the  price  down,  this  would  lower 
revenue  without  necessarily  reflecting  a  productivity  decline). 

Marketing 

Effective  organizations  perform  market  analyses  to  understand  their  markets  for  their  products 
or  services.  Market  analyses  provide  information  such  as  who  the  consumers  are,  where  they  are 
located,  why  they  purchase  the  product  or  service,  where  they  purchase  the  product  or  service,  what 
percentage  of  the  market  does  the  product  attract,  who  the  competition  is,  and  what  the  trends  in 
product  changes  are? 

Once  an  organization  has  determined  its  current  market  share,  it  can  set  goals  for  increasing  its 
share  and  explore  methods  for  doing  this.  Ordinarily,  we  don’t  think  of  a  public  sector  organization 
being  concerned  with  market  share,  competition,  or  the  generation  of  revenue.  However,  political 
trends  are  encouraging  the  public  sector  to  behave  more  like  the  private  sector.  Marketing  is 
especially  critical  for  organizations  pursuing  PGS  because  new  markets  represent  new 
opportunities  to  expand  and  improve.  A  useful  measure  related  to  marketing  is: 

7.  Market  Share:  The  amount  of  products  and  services  provided  by  the  organization  as  a 
percentage  of  the  total  products  and  services  available  in  the  market.  For  example,  a  research  and 
development  organization  can  assess  what  percent  of  the  government  research  dollars  in  their 
specialty  area  they  currently  attract.  If  they  can  develop  strategies  for  increasing  their  market  share, 
they  can  enhance  their  reputation,  attract  more  highly  qualified  researchers,  and  perform  higher 
quality  research.  Percent  of  market  share  thus  becomes  a  measure  of  success,  with  improvement 
indexed  by  a  higher  market  share. 


17 


Stakeholder  Relations 


The  third  major  area  identified  in  Table  1  is  stakeholder  relations.  Stakeholders  are  groups  and 
individuals  inside  and  outside  the  organization  who  affect  and  are  affected  by  the  organization. 
Stakeholder  relations  refers  to  the  effectiveness  of  an  organization  in  satisfying  its  stakeholders  and 
insuring  the  organization’s  long-term  survival.  The  payoff  for  improving  st^eholder  relations  is 
not  immediate;  rather  improving  stakeholder  relations  helps  to  secure  a  future  for  the  organization. 
Stakeholder  relations  can  be  subdivided  into  four  general  areas:  employee  quality  of  work  life 
(QWL),  customer  relations,  public  relations,  and  government  relations. 

Employee  Quality  of  Work  Life 

QWL  addresses  how  the  relationship  between  individuals  and  features  of  their  physical,  social, 
and  economic  work  environment  affects  on-and-off-the-job  attitudes  and  behaviors  (Wood, 
Rasmussen,  &  Lawler,  1975,  p.  23).  Employee  QWL  can  be  measured  in  different  ways  for 
different  attributes.  Some  measures  of  employee  QWL  are: 

1.  Organizational  Climate  Survey:  Such  a  survey  assesses  different  areas  of  QWL.  Typically, 
a  climate  survey  contains  sections  that  cover  job  satisfaction,  stress,  motivation,  commitment,  and 
organizational  culture.  Each  of  these  sections  can  usually  be  administered  separately  or  together  to 
obtain  a  general  picture  of  these  facets  of  QWL.  The  Navy  Personnel  Research  and  Development 
Center  has  available  a  climate  survey  that  will  reliably  and  accurately  measure  this  characteristic 
of  an  organization.  If  a  climate  survey  is  administered  on  a  regular  basis,  improvements  in  these 
areas  can  be  shown  across  time. 

2.  Data  From  the  Dispensary:  The  Dispensary  should  have  records  on  the  illnesses,  accidents, 
and  injuries  that  have  been  filed.  These  can  be  measured  by  looking  at  incidence,  frequency,  and 
severity  rates.  These  can  then  be  categorized  into  insured  and  uninsured  illnesses,  accidents,  and 
injuries.  For  more  information,  see  Novit  (1979,  pp.  207-209).  Data  from  each  of  these  categories 
can  then  be  compared  against  past  organizational  records  and/or  to  other  comparable  organizations 
to  determine  if  improvements  are  being  made. 

3.  Absenteeism  and  Turnover:  The  personnel  office  should  have  records  of  employee 
absenteeism  and  turnover.  By  collecting  and  analyzing  this  information  periodically,  an 
organization  can  see  improvements  that  occur  over  time  (i.e.,  if  the  frequency  of  absenteeism  and 
turnover  diminishes  over  time,  this  shows  improvement). 

4.  Exit  Interviews:  Information  obtained  from  the  exit  interviews  can  be  classified  according 
to  reasons  for  leaving.  These  categories  can  by  tracked  over  time.  If  the  frequency  of  certain 
categories  decreases  over  time  (e.g.,  employees  left  because  of  poor  advancement  potential  or 
because  of  bad  working  conditions)  then  this  reveals  improvement  in  the  area  of  turnover. 

Customer  Relations 

Customer  relations  refers  to  the  relationships  an  organization  has  with  its  internal  and  external 
customers.  Internal  customers  are  members  within  the  organization  that  depend  on  one  another  in 
the  manufacturing  of  a  product  or  the  delivery  of  a  service.  External  customers  are  people  who  buy 
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or  use  the  organization’s  product  or  service.  Effective  organizations  listen  to  their  customers  wants, 
are  customer  oriented,  and  satisfy  the  requirements  of  the  customer  without  ignoring  price,  profit, 
quality,  and  cost  control  (Ishikawa,  1985). 

Both  internal  and  external  customers  have  needs  and  preferences  that  must  be  satisfied. 
Satisfaction  levels  of  internal  customers  can  be  assessed  by  a  survey.  Satisfaction  of  external 
customers  was  discussed  earlier  in  the  context  of  output  quality,  and  referred  to  the  continual  usage 
of,  purchase  of,  and  confidence  in  a  product  or  service  by  the  users.  In  the  context  of  customer 
relations,  the  interest  is  in  the  customer’s  perceptions  of  the  organization  as  a  whole  (its  reputation) 
and  not  just  the  quality  of  its  products  and  services.  Measures  of  customer  relations  include: 

1.  Internal  Customer  Survey:  Every  organization  is  composed  of  numerous  internal  supplier/ 
customer  relationships  (e.g.,  engineering  is  a  supplier  to  manufacturing,  manufacturing  is  a 
customer  of  engineering  but  a  supplier  to  sales).  A  survey  can  measure  the  degree  of  satisfaction 
for  these  internal  relationships  and,  if  these  measures  are  followed  over  time,  the  organization  can 
tell  if  improvements  are  occurring. 

2.  Organizational  Reputation  Surveys:  The  external  customers  of  the  organization  can  be 
surveyed  to  determine  their  perceptions  of  the  organization.  Questions  on  the  survey  can  address 
issues  related  to  the  organization’s  general  reputation  and  status  in  the  community.  If  the  survey  is 
administered,  on  several  different  occasions,  trends  can  be  noted  and  the  data  will  indicate  if  the 
organizations  reputation  is  improving  over  time. 

3.  Focus  Group  Interviews:  A  common  technique  used  in  marketing  involves  conducting 
interviews  with  focus  groups.  Focus  groups  are  small  groups  of  people  who  have  an  interest  in  the 
organization  or  its  products  and  services  (e.g.,  employees,  customers).  These  focus  groups  are 
interviewed  together  to  get  an  in-depth  analysis  of  the  organization  (e.g.,  what  is  good  or  bad  about 
the  organization,  what  areas  need  improvement,  what  problems  are  occurring?).  Generally,  the 
results  from  these  interviews  are  of  a  qualitative  nature  (e.g.,  verbatim  comments  from  members 
of  the  focus  group),  but  these  qualitative  data  can  be  quantified  using  content  analyses  or 
paralinguistic  process  analysis  (Dooley,  1984,  pp.  89-91).  For  example,  a  content  analysis  can 
show  the  extent  to  which  customers  use  positive  words  and  phrases  to  describe  the  organization.  If 
focus  group  interviews  are  conducted  periodically,  content  analysis  might  show  a  trend  toward 
more  positive  descriptions.  Such  a  trend  would  be  indicative  of  increased  positive  perceptions  from 
the  customers  and  an  improved  reputation. 

4.  Unsolicited  Letters:  Many  organizations  (especially  those  involved  in  delivering  services) 
receive  unsolicited  letters  of  either  condemnation  or  commendation.  These  comments  can  be 
classified  (e.g.,  as  positive  or  negative)  and  the  changes  can  be  viewed  across  time.  An  increase  in 
positive  letters  or  a  decrease  in  negative  letters  are  indications  of  improved  customer  relations. 

Public  Relations 

Organizations  must  be  concerned  with  their  image  and  reputation  in  the  eyes  of  all  their 
stakeholders  (e.g.,  sponsors,  citizens,  taxpayers,  shareholders,  owners).  Private  organizations  are 
especially  concerned  with  how  they  are  perceived  by  their  shareholders  and  owners.  Public 
organizations  are  more  concerned  with  the  perceptions  of  legislators,  sponsors,  citizens,  and 
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taxpayers  (who  are,  in  a  sense,  like  voting  shareholders  in  a  corporation).  The  U.  S.  Postal  Service, 
the  Social  Security  Administration,  the  Internal  Revenue  Service,  and  the  DOD,  for  example,  must 
be  concerned  with  the  image  they  project  to  public  officials  and  the  general  public.  In  addition, 
government  organizations  must  cultivate  positive  relations  with  their  claimants  and  sponsors 
because  funding  (and  sometimes  their  survival)  depends  on  these  outside  organizations.  Without  a 
good  image  and  a  sound  reputation,  long  term  survival  is  in  jeopardy.  A  positive  image  aids 
survival  by  attracting  and  maintaining  funding,  keeping  current  customers  content,  and  building  a 
base  of  support  from  the  taxpayers.  An  organization’s  image  can  be  influenced  by  at  least  four 
things:  (1)  the  outcomes  of  its  efforts;  (2)  the  way  it  conducts  business;  (3)  the  effect  it  has  on  the 
environment,  and  (4)  how  well  it  serves  the  community.  Measuring  and  improving  public  relations 
can  be  accomplished  in  several  ways: 

1.  Public  Opinion  Survey:  Various  stakeholder  groups  can  be  surveyed  to  determine  if  they 
have  positive  or  negative  attitudes  about  the  organization.  Questions  might  include:  How  positive 
do  you  perceive  us  in  comparison  to  other  similar  organizations?  Do  you  perceive  our  organization 
as  one  that  can  be  trusted  to  provide  quality  products?  Do  you  view  our  business  actions  in  a 
positive  manner?  Do  you  feel  that  our  organization  affects  the  environment  in  a  negative  manner? 
Do  you  feel  that  our  organization  is  an  asset  to  the  community?  This  information  can  be  used  to 
address  deficiencies  in  your  public  image  and  to  fortify  the  things  you  are  doing  well. 
Improvements  will  be  revealed  by  higher  ratings  of  your  organization  by  the  various  groups. 

2.  Unsolicited  Letters  of  Approval  or  Disapproval:  The  organization  can  measure  how 
positive  its  public  image  is  by  the  number  of  unsolicited  letters  of  approval  or  disapproval.  The 
content  of  these  letters  can  be  classified  as  positive  or  negative  and  followed  over  time.  Improved 
public  relations  is  revealed  by  an  increase  in  positive  letters  and  a  decrease  in  negative  letters. 

Government  Relations 

All  organizations  must  comply  with  government  regulations.  Specific  agencies  ensure  that 
organizations  complying  with  regulations  regarding  their  type  of  business.  For  instance,  the  Equal 
Employment  Opportunity  Commission  has  regulations  that  govern  the  hiring  and  firing  of 
employees.  If  organizations  fail  to  comply  with  these  regulations,  the  government  has  the  legal 
authority  to  administer  penalties,  fines,  and  even  to  close  them  down.  Some  alternative  ways  of 
measuring  effective  government  relations  include: 

1.  Inspection  Team  Reports:  Many  organizations  undergo  periodic  inspections  to  insure  that 
they  are  complying  with  local,  state,  or  federal  regulations.  An  organization  can  compare  how  well 
it  was  assessed  by  the  inspection  team  (e.g.,  number  of  discrepancies  weighted  by  the  seriousness 
of  each)  to  previous  years.  Li  this  way,  they  can  measure  how  much  they  have  improved. 

2.  Complaints  Regarding  Regulatory  Violations:  The  organization  can  measure  how  well  it  is 
doing  in  different  areas  by  how  many  grievances  or  complaints  (by  employees,  unions,  regulatory 
agencies)  have  been  filed  during  a  specific  time.  These  complaints  can  then  be  compared  to 
previous  years.  This  will  give  an  indication  of  how  well  the  organization  is  complying  with  various 
regulations  (e.g.,  equal  employment,  fraud,  waste,  abuse). 
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3.  Meeting  Codes/Regulations:  Organizations  can  document  their  compliance  with  various 
codes  and/or  regulations.  For  example,  they  can  document  all  compliance  with  fire  code 
specifications.  They  can  also  document  whether  they  have  been  fined  or  penalized.  If  compliance 
tends  to  go  up  over  time,  and  fines  and  penalties  go  down,  this  serves  as  an  indication  of 
improvement  in  these  areas.  These  improvements  are  also  important  because  noncompliance  with 
codes  and  regulations  can  be  costly. 

Resource  Development 

The  fourth  area  in  Table  1  is  resource  development.  Measures  of  resource  development  are 
important  to  an  organizations  effectiveness  because,  before  an  organization  can  be  efficient,  it  must 
develop  the  quality  of  its  resources.  In  addition  to  increasing  efficiency,  developing  quality 
resources  creates  more  opportunities  to  improve  the  quality  of  products  and  services.  Resource 
development  can  be  subdivided  into  two  general  areas:  employee  development  and  technology 
development.  Each  of  these  areas  can,  in  turn,  be  broken  down  into  smaller  areas.  For  each  of  these 
subareas,  several  alternative  measures  will  be  suggested. 

Employee  Development 

Most  organizations  feel  that  the  employees  are  their  most  valuable  resource.  Certainly,  any 
organization  that  wants  to  deliver  quality  goods  and  services  needs  a  highly  skilled  and  dedicated 
work  force.  How  can  an  organization  tell  if  it  is  adequately  developing  its  human  talent?  It  must 
measure  important  areas  of  employee  development  and  then  make  efforts  to  improve  in  these  areas. 
Employee  development  consists  of  three  major  areas:  (1)  recruitment,  (2)  selection,  and  (3) 
training.  We  will  provide  a  brief  review  of  these  areas  and  then  suggest  ways  of  determining 
whether  an  organization  is  doing  a  good  job  in  developing  its  employees. 

Recruitment 

Employee  development  requires  that  an  organization  have  some  notion  of  its  manpower  needs 
and  the  size  of  the  labor  market.  Knowing  manpower  needs  and  predicting  labor  supply  is  crucial 
to:  (1)  succession  planning,  (2)  strategic  planning,  (3)  reductions  in  work  force,  (4)  budgeting,  and 
(5)  formulating  training  and  recruitment  strategies.  The  success  of  recruitment  is  measured  in 
terms  of  the  recruitment  ratio.  The  recruitment  ratio  is  the  ratio  of  the  number  of  persons  recruited 
to  the  numbered  of  qualified  candidates.  Generally,  an  organization  wants  a  high  selection  ratio 
(i.e.,  a  large  number  of  qualified  candidates  for  a  given  number  of  recruits)  because  this  reflects  an 
efficient  use  of  resources.  An  organization  can  use  one  or  more  of  the  following  sources  to  obtain 
data  for  computing  a  recruitment  ratio. 

1.  Data  From  Personnel  Office:  The  personnel  office  should  have  records  on  who  was 
considered  for  each  position  and  who  met  the  minimum  qualifications.  A  ratio  can  be  computed 
from  these  data  that  reflects  the  proportion  of  job  applicants  to  qualified  candidates.  Sorting 
through  the  applications  of  unqualified,  or  marginally  qualified,  candidates  wastes  time  and 
energy.  If  computing  the  ratio  over  time  yields  an  upward  trend,  then  the  recruitment  process  is 
likely  to  improve  (i.e.,  the  organization  is  recruiting  more  qualified  candidates). 
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2.  Data  From  Bureau  of  Labor  Statistics  and  Census  Bureau:  This  information  can  tell  the 
organization  the  condition  of  the  national  and  regional  labor  market  for  various  labor  categories. 
The  organization  can  then  compare  labor  categories  within  the  organization  to  these  national  and 
regional  demographics  to  determine  whether  their  recruitment  policies  are  better  or  worse  than 
national  and  regional  trends  (for  an  example  of  how  such  data  was  used  in  the  U.S.  Navy,  see 
Atwater  &  Niehaus,  1993). 

3.  Data  From  Local  Information  Sources:  Information  on  the  local  labor  supply  can  be 
gleaned  from  the  Chamber  of  Commerce,  local  newspapers,  or  employment  agencies.  This 
information  can  be  useful  in  assessing  recruitment  efforts  by  comparing  the  labor  situation  within 
the  organization  (e.g.,  EEO  complaints,  union  grievances,  level  of  diversity)  with  the  situation  in 
the  local  community.  Again,  trends  across  time  show  the  organization  whether  improvements  are 
being  made  or  whether  corrective  action  is  needed. 

Selection 

Selection  refers  to  hiring  the  right  person  for  the  job.  Better  selection  will  improve  an 
organization’s  performance  between:  (1)  employees  who  have  the  right  abilities  perform  more 
effectively,  (2)  good  selection  procedures  lower  costs  (e.g.,  if  people  are  retained  longer,  the  cost 
of  recruitment  and  new-employee  training  goes  down),  (3)  good  selection  avoids  many  legal 
problems  resulting  from  unfair  hiring  procedures  (Dessler,  1988,  pp.  167-168). 

Improving  the  selection  process  is  complicated  and  requires  information  on  the  (1)  cost  of  the 
selection  tests,  (2)  validity  and  reliability  of  the  tests,  (3)  costs  of  recruiting  and  training, 
(4)  selection  ratio  (the  proportion  of  the  candidates  that  are  selected),  and  (5)  estimates  of  the 
employees’  dollar  contribution  to  the  organization.  With  this  information,  there  are  several 
techniques  for  conducting  what  is  known  as  “utility  analysis”  (e.g.,  Cascio  &  Ramos,  1986; 
Schmidt,  Hunter,  McKenzie,  &  Muldrow,  1979).  A  utility  analysis  of  an  organization’s  selection 
procedures  will  show,  in  dollars,  the  effectiveness  of  the  current  procedures.  A  utility  analysis  can 
also  be  used  as  a  metric  to  measure  the  degree  of  improvement  resulting  from  changes  in  the 
selection  procedures.  For  example,  if  the  organization  adopts  a  new,  more  expensive  selection  test 
with  a  higher  validity,  a  utility  analysis  can  show  whether  the  new  test  is  cost  effective.  The 
information  needed  for  utility  analysis  will  come  from  several  sources. 

1.  Data  From  Personnel  Office:  The  personnel  office  should  have  records  on  who  was 
considered  for  each  job  position  and  who  was  hired.  A  selection  ratio  can  be  computed  that  reflects 
the  proportion  of  job  candidates  to  job  hires.  The  personnel  department  should  also  have 
information  on  the  different  selection  tests  employed  and  their  costs,  rehabilities,  and  validities. 
Finally,  data  on  the  costs  of  recruiting  and  training  new  employees  should  be  available  through  the 
personnel  office. 

2.  Survey  of  Budget  Analysts:  Budget  analysts,  or  others  with  expertise  in  the  financial 
performance  of  the  organization,  must  be  surveyed  to  obtain  data  on  the  differences  between  the 
value  of  the  products  and  services  produced  by  different  employees. 
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Training 


How  can  an  organization  tell  if  its  training  program  is  effective?  In  order  to  have  coordinated, 
methodical,  cohesive,  and  accountable  training  programs  that  share  symmetry  with  the 
organization’s  objectives,  a  needs  assessment  and  evaluation  of  training  effectiveness  must  be 
done.  To  know  if  the  training  is  effective,  the  organization  must  have  some  idea  of  its  critical  needs. 
Once  its  needs  are  known,  the  organization  can  then  design  a  training  program  to  meet  those  needs. 
Effective  training  programs  increase  the  knowledge,  skills,  and  abilities  of  employees  in  those 
areas  of  critical  need,  ff  an  organization  is  using  its  resources  wisely,  then  it  should  be  able  to  show 
improvements  in  training  effectiveness  over  time.  The  effectiveness  of  the  training  program  can  be 
assessed  in  a  number  of  ways. 

1.  Acquisition  Measures:  The  effectiveness  of  training  in  the  acquisition  of  knowledge,  skills, 
and  abilities  can  be  measured  by  giving  the  trainee  a  test  before  and  after  training.  The  difference 
in  the  before-and-after  test  scores  will  yield  information  regarding  newly  acquired  knowledge  and 
skills.  Such  tests  can  be  created  in-house,  or  if  an  outside  organization  conducts  the  training,  an 
external  agency  may  provide  the  test  results.  The  organization  should  continuously  collect  this 
information  and  see  if  the  acquisition  of  skills  and  knowledge  improves  over  time.  Improvements 
in  these  acquisition  measures  would  reflect  improvements  in  training  efforts. 

2.  Job  Performance  Measures:  Effective  training  involves  not  only  acquisition  of  new 
knowledge,  skills,  and  abilities,  but  also  the  changes  in  actual  job  performance.  Measures  of  job 
performance  can  be  assessed  before  and  after  the  training.  If  the  employee  can  improve  job 
performance  over  time,  it  is  an  indication  that  the  training  is  effective. 

3.  Standardized  Tests:  A  measure  of  the  quality  of  your  training  can  be  obtained  by 
administering  standardized  tests  that  assess  the  knowledge,  skills,  and  abilities  that  you  desire  in 
your  work  force.  These  tests  are  available  through  a  large  number  of  commercial  publishers,  but 
are  usually  expensive  and  should  be  administered  by  a  qualified  examiner.  However,  should  you 
choose  to  go  this  route,  you  can  track  improvements  over  time  and  assess  the  effectiveness  of  your 
training. 

Technology  Development 

Technology  development  refers  to  the  use  of  resources  for  development  of  new  technology  in 
the  workplace.  It  consists  of  the  physical  and  informational  resources  by  which  people  bring  about 
some  desired  result  (e.g.  manufacturing  a  product  or  delivering  a  service).  Technology 
development  concerns  the  methods,  processes,  and  physical  resources  an  organization  uses  in  the 
course  of  business. 

Work  Methods 

Organizations  should  have  a  thorough  understanding  of  the  methods  and  processes  that  are 
used  to  create  their  products  or  deliver  their  services.  First,  an  organization  should  identify  the 
types  of  work  methods  that  must  be  performed.  Second,  the  organization  should  select  the  specific 
work  methods  best  suited  to  its  operation.  Finally,  the  organization  should  assess  the  effectiveness 
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of  its  work  methods.  Work  methods  that  are  ineffective  should  either  be  improved  or  replaced. 
Several  possible  measures  of  work  methods  are: 

1.  Time  Since  Last  Process  Analysis:  Process  analyses  determine  the  best  work  methods  for  a 
particular  operation.  If  it  has  been  a  long  time  since  such  a  study  has  been  performed,  this  could  be 
an  indication  that  the  most  up-to-date  methods  are  not  being  employed.  An  organization  should 
keep  track  of  the  frequency  with  which  such  methods  and  analysis  studies  are  conducted,  and 
compare  the  data  to  past  performance  periods.  If  the  time  lag  between  studies  grows  too  large 
relative  to  other  periods,  this  may  be  an  indication  that  the  work  methods  are  not  being  managed 
well. 

2.  Benchmarking:  This  technique  allows  the  organizations  to  compare  the  effectiveness  of  its 
work  methods  to  the  effectiveness  of  methods  used  at  similar  organizations  that  are  considered  to 
be  the  ‘'best”  organizations.  An  organization  can  form  a  ratio  of  the  performance  of  their  methods 
to  the  performance  of  the  methods  used  by  other  organizations.  When  this  ratio  is  observed  over 
time,  the  organization  can  see  whether  their  methods  are  improving,  and  if  not,  what  new  work 
methods  might  be  appropriate. 

3.  Use  of  Latest  Technology:  If  an  organization  is  familiar  with  the  latest  technology,  it  can 
record  what  percentage  of  its  current  work  methods  conform  to  this  technology.  For  example,  a 
software  developer  can  record  the  use  of  current  programming  tools,  and  determine  the  percentage 
of  these  tools  that  conform  to  the  most  recent  technical  tools  available  (e.g.,  CASE  tools,  object 
oriented  programming).  As  the  percentage  increases,  the  organization  can  infer  that  it  is  improving 
work  methods. 

Physical  Resources 

The  physical  resources  of  an  organization  are  its  facilities  and  tools  and  equipment.  How  does 
an  organization  know  if  it  has  the  proper  physical  resources  for  its  operation?  There  are  several 
aspects  that  an  organization  should  be  concerned  with. 

1.  Age  of  Physical  Resources:  Organizations  usually  keep  records  on  the  purchase  dates  of 
tools,  equipment,  and  facilities.  These  records  tell  how  old  these  resources  are.  As  the  tools, 
equipment,  and  facilities  age,  the  resources  become  less  useful  and  eventually  require  replacement 
The  age  relative  to  useful  life  can  be  used  as  an  index  of  how  effective  the  resources  are  being  used. 
If  the  average,  age  of  the  resources  decreases  over  time,  this  tells  the  organization  that  it  is  keeping 
the  tools,  equipment,  and  facilities  up  to  date. 

2.  Condition  of  Physical  Resources:  Most  organizations  keep  maintenance  records  on  their 
tools,  equipment,  and  facilities.  These  records  can  be  used  to  show  the  physical  condition  and 
operational  state  of  the  resources.  From  these  records,  an  index  can  be  developed  that  can  be 
compared  from  one  time  period  to  the  next.  If  the  index  improves  over  time,  the  organization  shows 
improvement  in  the  use  of  its  physical  resources. 

The  success  of  PGS  rests  on  the  quality  of  the  measurement  system.  The  measurement  system 
must,  of  course,  accurately  reflect  improvements  in  productivity. 
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Conclusions 


Measures  of  productivity  are  only  part  of  the  picture  of  productivity  gain  sharing.  PGS  is  a 
systems  approach  to  organizations,  and  therefore  the  emphasis  should  be  on  total  organizational 
improvement  and  not  just  improvement  in  one  area.  This  report  has  been  a  primer  and  a  brief 
introduction  to  organizational  effectiveness.  We  have  shown  that  effectiveness  can  be  divided  into 
four  distinct  areas  (productivity,  financial  performance,  stakeholder  relations,  and  resource 
development),  and  we  have  given  many  examples  of  how  each  of  these  areas  can  be  measured. 
Although  the  examples  come  primarily  from  white  collar  organizations,  most  of  the  measures 
discussed  apply  to  all  types  of  organizations.  The  important  point  is  that  an  effective  PGS  plan 
should  incorporate  some  measures  from  all  four  areas  so  that  total  organizational  performance  can 
be  monitored  and  improved.  Effective  organizations  realize  that  there  must  be  a  balance  between 
these  areas.  If  one  aspect  of  the  organization  is  performing  optimally  (e.g.,  human  resources  are 
being  effectively  utilized),  but  another  area  is  performing  poorly  (e.g.,  the  budget  can  not  be 
adhered  to),  then  the  overall  ability  of  the  organization  to  perform  will  be  compromised. 

Reviewing  the  measures  set  forth  in  this  report  provides  the  organization  with  some  tools  for 
assessing  where  its  strengths  and  weaknesses  lie.  Reviewing  these  measures  should  cause  an 
organization  to  think  about  areas  not  previously  attended  to  that  are  important  to  achieving  the 
organization’s  goals.  We  do  not  wish  to  convey  the  idea  that  every  organization  should  adopt  all 
of  these  proposed  measures,  nor  do  we  believe  that  this  document  contains  an  exhaustive  list  of 
performance  measures.  Each  organization  should  decide  which  measures  are  most  relevant  to  its 
goals  and  needs,  and  customize  and  develop  its  own  measures  if  these  do  not  suit  its  purposes.  If 
an  organization  does  develop  its  own  performance  measures,  we  hope  that  the  examples  presented 
here  serve  as  a  useful  guide.  At  the  very  least,  an  organization  should  select  or  develop  measures 
from  each  of  the  four  major  areas  (productivity,  financial  performance,  stakeholder  relations,  and 
resource  development). 

Of  course,  selecting  the  relevant  set  of  measures  is  just  the  beginning  as  far  as  PGS  is 
concerned.  These  measures  must  also  be  monitored  over  time,  revised,  adjusted,  and  replaced.  The 
set  of  measures  must,  at  some  point,  be  combined  to  form  a  unified  picture  of  total  organizational 
performance.  This  combining  of  performance  measures  is  discussed  at  some  length  by  Nebeker 
and  Tatum  (1996)  and  Tatum,  Nebeker,  and  De  Young  (1996)  who  demonstrate  alternative 
approaches  to  aggregating  performance  measures.  Finally,  these  measures  must  be  translated  into 
dollar  values  so  that  appropriate  financial  payouts  can  be  made  based  on  the  value  of  the  measured 
improvements  in  the  organization  (see  the  two  Nebeker  and  Tatum  articles  for  suggestions  on  how 
this  translation  can  be  accomplished). 
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